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[57] ABSTRACT 

A laser-directed ranging system for use in telerobotics 
applications and other applications involving physically 
handicapped individuals. The ranging system includes a left 
and right video camera mounted on a camera platform, and 
a remotely positioned operator. The position of the camera 
platform is controlled by three servo motors to orient the roll 
axis, pitch axis and yaw axis of the video cameras, based 
upon an operator input such as head motion. A laser is 
provided between the left and right video camera and is 
directed by the user to point to a target device. The images 
produced by the left and right video cameras are processed 
to eliminate all background images except for the spot 
created by the laser. This processing is performed by creat- 
ing a digital image of the target prior to illumination by the 
laser, and then eliminating common pixels from the subse- 
quent digital image which includes the laser spot The 
horizontal disparity between the two processed images is 
calculated for use in a stereometric ranging analysis from 
which range is determined. 

37 Claims, 6 Drawing Sheets 
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RANGING APPARATUS AND METHOD 
IMPLEMENTING STEREO VISION SYSTEM 

ORIGIN OF THE INVENTION 

The invention described herein was made in the perfor- 
mance of work under a NASA contract and is subject to the 
provisions of Section 305 of the National Aeronautics and 
Space Act of 1958, Public Law 85-568 (72 Stat 435; 42 
U.S.C. 2457). 

BACKGROUND OF THE INVENTION 

The present invention relates generally to systems for 
determining the range of an object from a reference point, 
and more particularly to ranging systems useful in robotic 
and other applications, such as surveying, production lines, 
and tiie like. Still more particularly, the present invention 
relates to laser-directed ranging systems useful in telerobot- 
ics applications. 

Ranging techniques are generally classified into one of 
two major categories — either active or passive. Active rang- 
ing techniques are those that actively engage the target by 
introducing an effect not already a part of the local 
environment, as for example, by electromagnetic radiation, 
sonic waves, or a laser beam One common active ranging 
technique is based on “time-of-flight” detection, in which 
the transmission time of a radiated energy signal 
(electromagnetic, sonic, or ultrasonic radiation) to and from 
the target is measured and the range thereby calculated. 

A second common active ranging technique relies on 
phase-shift detection in a reflected signal Sometimes called 
tiie Moire method, one amplitude-modulated spatial signal 
(e.g., reflected light from a scene) is multiplied by another 
amplitude-modulated spatial signal (e.g., a viewing grating) 
to create an output signal with surface depth infannation 
encoded as a phase difference. 

In holographic interferometry, a third active ranging 
technique, coherent light from two separate laser beams, 
focused at a common surface point, is added and the surface 
depth is encoded in the detected phase difference. 

Triangulation ranging techniques actively measure two 
interior angles, angle AB and angle BC, and the baseline B 
of a triangle ABC and then determine the length A or C from 
the viewing apparatus to tiie target surface. Basically, either 
the ambient light reflected from tiie target surface may be 
viewed from two angles, on opposite ends of the baseline, or 
light may be projected onto the target surface from one end 
of the baseline and viewed or detected from the opposite end 
of the baseline. 

Passive ranging techniques measure tiie distance between 
a target and a fixed reference point on the basis of informa- 
tion derived from the target without illuminating or irradi- 
ating the target. Stadimetry, for example, determines the 
distance to a target from the known height of a target feature 
and the size of that feature within the image (Le., the angle 
subtended by that target feature in the viewed image). This 
technique requires that the size of the target feature be 
known in order to calculate the target range. 

Another passive ranging technique is that employed by 
the traditional parallax range finder. In this technique, tiie 
target is viewed from two optical apertures. When the lines 
of sight of the two apertures are parallel, two images are 
observed. By varying the angle of one viewing aperture, 
coincidence of tiie images is effected and angular variation 
of the one viewing aperture is a measure of target range. 

Stereometric ranging is yet another passive ranging tech- 
nique utilizing a stereoscopic image pair to deduce the 


2 

three-dimensional characteristics of a target In stereoscopic 
imaging, a pair of cameras are arranged as shown in FIG. 1, 
with the line-of-sight of the two cameras parallel, lying in 
the horizontal plane, and separated by distance, b. The 
5 normal range, z, from the front of the left camera lens to the 
vertical plane defined by a point, p, is calculated as follows: 

Z ~ *\~*r 

10 

where f is the focal length of the two cameras, and x ; and x r 
are the horizontal coordinates of the image of point, p, in the 
left and right camera images, respectively. The difference 
(x,-— x r ) is sometimes referred to as the “disparity.” 

15 Telerobotics is a collection of technologies that enable a 
human operator to control a robot at a remote site. A suitable 
ranging technique is an essential element of telerobotics 
applications to enable the remote operator to direct a robot 
manipulator to a particular position to effect a particular 
20 operation relative to a target Many of the conventional 
ranging techniques are not suitable for telerobotics applica- 
tions. Relatively inexpensive radar or sonar ranging instru- 
ments are incapable of reliably and consistently discrimi- 
nating range in environments that include a number of 
25 objects having differing size and specular reflection charac- 
teristics. More accurate time-of-flight instruments are 
expensive. Laser and other phase change detectors also are 
quite expensive and are bulky and consume large amounts of 
power. 

30 As for passive ranging techniques, stadimetry, requiring 
advance knowledge of the height of a target, is not suitable 
for real-time telerobotics applications. Accordingly, most 
telerobotics applications utilize some form of parallax or 
stereometric ranging. These techniques, however, require for 
35 their success a high-contrast environment for enhanced 
accuracy and significant computational power, because the 
entire field of view of the camera images must be processed 
to identify the target within the image and to compute either 
the point of coincidence (for parallax ranging) or the dis- 
40 parity (for stereometric ranging). The computations typi- 
cally involve large numbers of convolution and cross- 
correlation algorithms and result in a majority of time being 
spent processing unwanted range information. Because of 
the complex algorithms, these techniques are not suitable for 
45 real-time applications without using expensive, high-speed 
computers. 

Hence, it would be advantageous to develop a ranging 
technique suitable for use in real time telerobotic applica- 
tions that does not require the processing of complex algo- 
50 rithms and thus does not require the use of expensive, 
high-speed computers. 

SUMMARY OF THE INVENTION 

Accordingly, there is provided herein a laser directed 
55 ranging system, adapted for telerobotics applications, in 
which a frame grabbing system connected to stereo video 
cameras, and a processor operating in accordance with a 
disparity measurement algorithm, are operable for determin- 
ing the range to a localized area or spot within the field of 
60 view of the cameras which is illuminated by a directional 
light source, preferably a laser in the visible or infrared 
range, such as a laser diode ruby red laser source, in fixed 
alignment with the cameras. 

The video cameras are mounted on a camera platform 
65 which can be positioned remotely by an operator. In the 
preferred embodiment, position and orientation sensors are 
provided to determine head movements of the operator, and 
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to produce electrical signals indicative thereof. The electri- 
cal signals indicative of the operator’s head movements are 
provided to the camera platform to position the platform 
accordingly. 

Once the video cameras are positioned properly, the 
operator activates a laser, or other directional light source, to 
produce a spot on the target device. The processor then shuts 
off the laser momentarily so that a left and right video image 
of the target device can be obtained without the laser spot 
The laser then is turned on and a second set of left and right 
video images are taken of the target device with the laser 
spot The video images are digitized for further processing. 

The processor next generates a left and right image of the 
laser spot by eliminating common pixels between the 
respective image of the target device without the spot and 
the respective image of the target device with the spot The 
processor then determines the disparity between the left and 
right image of the laser spot This disparity value then is 
used to determine the range to the target device in a 
stereometric ranging process. After the range to the target 
device is determined, the processor can position a robotics 
arm to manipulate the target device as desired 

BRIEF DESCRIPTION OF THE DRAWINGS 

For a more detailed description of the preferred embodi- 
ment of the invention, reference will now be made to the 
accompanying drawings, wherein: 

FIG. 1 is a diagram that demonstrates the prior art 
technique of stereometric ranging; 

FIG. 2 is a functional block diagram depicting a laser- 
directed ranging system constructed in accordance with the 
principles of the present invention; 

FIG. 3 is a more detailed block diagram showing the 
components comprising the laser-directed ranging system 
depicted in FIG. 2 and their interrelationship; 

FIG. 4 depicts successive images from the stereo cameras 
shown in FIG. 2 to illustrate the frame subtraction technique 
of the present invention; 

FIG. 5 is a flow chart describing the ranging algorithm 
executed by the computer shown in FIG. 2; 

FIG. 6 is a typical graph of the relationship between the 
stereometric image disparity and the range for the ranging 
system depicted in FIG. 2; 

FIG. 7 is a perspective drawing illustrating a robotics 
application for the ranging system of the present invention; 
and 

FIG. 8 is a perspective drawing illustrating an application 
for the present invention involving robotics assistance to 
handicapped individuals. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENT 

Of the many different active and passive ranging tech- 
niques currently available, none are particularly well suited 
to real time telerobotic applications. Inexpensive active 
ranging systems, such as radar and sonar systems, are not 
sufficiently discriminatory and accurate for telerobotic 
applications. More accurate “time-of-flight” and phase 
change type active ranging systems are expensive and 
typically too bulky for telerobotic applications. Passive 
techniques such as stereometric ranging require high con- 
trast environments and significant computational power to 
operate on a real-time basis. The present invention solves 
these problems by providing an improved stereometric rang- 
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ing system in which a low-power laser is used to superim- 
pose a high contrast image on the target and thereby to 
enable the elimination of unnecessary processing of back- 
ground images surrounding the illuminated spot on the 
5 target 

Referring now to FIG. 2, a laser-directed ranging system 
10 constructed in accordance with the principles of the 
present invention includes a stereo camera imaging and 
ranging system 12, a head-mounted display system 14, a 
to position and orientation sensor system 16, a laser/camera 
platform control system 18, and a robot control system 20. 

A. General Apparatus and Method of Operation 
Referring still to FIG. 2, the ranging system 10 is designed 

to permit an operator to control a robot 19 at a remote 
15 location with head motions and voice commands. The stereo 
camera imaging and ranging system 12, which is located at 
a location remote to the operator, captures visual images of 
the remote location. The visual images are conveyed elec- 
tronically to the operator and provide real time feedback to 
20 the operator’s head-mounted display system 14, whereby the 
operator can observe and direct the robot 
A laser 42 or other direction light source preferably 
mounts between the two cameras 30 in the stereo camera 
imaging and ranging system 12. In the preferred 
25 embodiment, the laser 42 aligns in a parallel plane with the 
video cameras 30. One skilled in the art will immediately 
realize, however, that the laser 42 may be offset with respect 
to the video cameras, so long as the laser spot hits and is 
reflected off the target The laser 42 enables the operator to 
30 illuminate a target. The video images, both laser-illuminated 
and not, are captured from the stereo cameras 30 in con- 
ventional frame grabber circuitry 34 and are used in a 
computer-based ranging algorithm to identify the position of 
a target relative to the position of the robot 19. 

The position and orientation sensor system 16 connects to 
the operator’s helmet and monitors the position and orien- 
tation of the operator’s head. The position and orientation 
data is delivered to the laser/camera platform control system 
18, which adjusts the position and orientation of the laser 
40 and cameras in conformance with the operator’s head move- 
ments. 

In the preferred operation, the operator turns on the laser 
42 by voice command and points the laser at the robot target 
by appropriate head movements. When the laser is properly 
45 targeted, the operator commands the system to identify the 
range to the target and commands the robot to engage the 
target as desired. In the sections that follow, the components 
preferably comprising the ranging system 10 are identified 
and their interrelationship and operation are explained in 
50 detail. 

B. Stereo Camera Imaging and Ranging System 
Referring now to FIG. 3, the stereo camera imaging and 

ranging system 12 comprises left and right video cameras 
30, 32, each connected through conventional video frame 
55 grabber boards 34, 36 to a general purpose computer 40, 
such as a personal computer or workstation, and a laser 
pointer 42 connected to the computer 40 through a current 
relay switch 44. 

The video cameras 30, 32 preferably are high quality 
60 color video cameras having an image discrimination capa- 
bility sufficient for the desired application of the ranging 
system, preferably with automatic gain control. One suitable 
camera type is the Model TMC-50 manufactured by Pulnix. 
The video cameras 30, 32 are mounted with their lines of 
65 sight preferably in parallel alignment on a platform, which 
is described more particularly below in the section entitled, 
“Laser/Camera Platform Control System.” 
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The left and right camera, 30, 32 are connected to the 
conventional video frame grabber boards 34, 36, which are 
received in slots in the chassis of the computer 40, For 
example, the inventors have used Datacube Max Video 20 
frame grabber boards with success in such a telerobotics 
application. The frame grabber boards capture a video image 
on command, digitize the image, and store the digitized 
image or transmit it on command to the computer 40. The 
computer 40 may be any microprocessor-based computer 
capable of executing the control and analysis algorithms 
described herein, including, for example, Model No. Sparc 
2 workstation manufactured by Sun Microsystems, Inc. 

The laser 42 preferably is a low power laser, such as a 1 
mW, class n ruby laser diode, which is relatively inexpen- 
sive and safe for human operating environments. A ranging 
algorithm, as described below, is executed by the computer 
40 and sets and resets a digital I/O bit to activate and 
deactivate the conventional current relay switch 44, which 
energizes and deenergizes the laser 42, all as is well known 
to persons having ordinary skill in the art of designing and 
implementing real time computer control systems. 

C. The Ranging Algorithm 

Applicants have devised a novel ranging algorithm that 
combines the use of a laser with conventional stereometric 
ranging theory to provide a ranging system that does not 
require processing of large numbers of convolution and 
cross-correlation algorithms. As a result. Applicant’s rang- 
ing algorithm can be used in real-time telerobotics applica- 
tions without a computer having enhanced processing capa- 
bilities. 

A sequence of video images for both the left and right 
cameras 30, 32, as shown in FIG. 4, will aid in understand- 
ing Applicants’ novel ranging algorithm. Referring now to 
FIGS. 3 and 4, Frame A and Frame B show representative 
images from the left and right cameras, respectively. Frames 
C and D represent images of the same scene from the two 
cameras, with the target box illuminated by the laser 42. 
Frames A and C are digitized and stored by the frame 
grabber board 34 for the left camera 30, as Frames B and D 
are digitized and stored by the frame grabber board 36 for 
the right camera 32. 

The computer 40 then compares Frame A with Frame C 
and creates a third image in which the common pixels from 
both frames are eliminated, leaving only the spot created by 
the laser illumination. In effect, the image of Frame A is 
subtracted from the image of Frame C. The same process is 
applied to Frames B and D to create an image of the laser 
spot from the right camera. The horizontal disparity between 
the spots in the two images can then be calculated, and the 
range to the target box can be calculated by conventional 
stereometric ranging analysis. 

Referring now to FIG. 5 for a more particular description 
of the ranging algorithm, the process begins by activating 
the laser in step 50. This can be initiated by a conventional 
manual switch connected to the computer or by the opera- 
tor’ s oral command, which is translated through a voice 
recognition system forming a part of the computer. Other 
techniques for activating the laser also can be used without 
departing from the principles of the present invention. 

In step 52, the algorithm waits for an indication from the 
operator that the operator is ready to take a range measure- 
ment Upon receiving the operator’s command to take a 
range measurement, the algorithm turns off the laser (step 
54) and then determines whether the camera is stable enough 
to execute the ranging measurement (step 56). This is 
accomplished, for example, by executing successive frame 
subtractions until the resulting difference for both cameras is 
a blank image for a predetermined length of time. 


When the camera is sufficiently steady, the algorithm then 
executes the frame subtraction routine as described above in 
connection with FIG. 4. Thus, each of the two frame grabber 
boards capture an image of the target (step 58), the laser is 
5 turned on (step 66), each of the two frame grabber boards 
capture an image of the target as illuminated by the laser 
(step 62), and the computer subtracts the non-illuminated 
image from the corresponding illuminated image to create a 
pair of images of the laser spot (step 64). 

10 Next, in step 66, the horizontal and vertical coordinates of 
the centroid of the laser spot in each of the two images can 
be calculated from the following formulas: 
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20 where, N represents the number of pixels comprising the 
laser spot after frame subtraction, and (x„, yj are the 
coordinates of these pixels in the image. If the video cameras 
are positioned in a common vertical plane, there will be no 
disparity in the vertical plane, so that only horizontal dis- 
25 parity need be determined. If the cameras are not in a 
common vertical plane, then vertical disparity also must be 
determined. 

The horizontal disparity and the range to the target are 
calculated in step 68. The horizontal disparity, d, which 
30 preferably is measured in pixels, is calculated as follows: 

d=x r x r 

where x / is the horizontal coordinate of the centroid of the 
laser spot as viewed through the left camera, and x r is the 
35 horizontal coordinate of the centroid of the laser spot as 
viewed through the right camera. 

The normal range, z, to the target then can be calculated 
as follows: 


40 


z 


=4 


where f is the focal length of the two cameras, and b is the 
baseline separation between the lines of sight of the parallel 
45 cameras. 

For improved accuracy, it may desirable to calibrate the 
range measurement empirically by determining the 
disparity, as described above, for a plurality of known 
ranges. Thus, one can set up the cameras with a target 
50 positioned on the centerline between the two cameras, move 
the cameras along the centerline to a plurality of predeter- 
mined target ranges, measuring the disparity at each range, 
generate a curve depicting range versus disparity, and apply 
a functional approximation technique, such as a least 
55 squares algorithm, to define the range as a function of 
disparity for the particular camera arrangement in use. The 
robotic arm can be used to move the target along the camera 
centerline for calibration through the implementation of a 
kinematics algorithm, which moves the robotic arm in 
6o Cartesian space in a “linear fashion”. A Kinematics algo- 
rithm typically is provided with a commercial robotics 
system such as the type PUMA 560 robotics manipulator 
manufactured by Staubli. 

FIG. 6 depicts a typical curve for an empirical measure- 
65 ment of the range versus disparity curve. If range is to be 
determined from an empirical calibration of the cameras, 
step 68 of the ranging algorithm (FIG. 5) becomes, instead 
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of a calculation as indicated above, a table look-up of range 
from measured disparity, or a calculation of range from 
disparity using the function that fits the empirically mea- 
sured curve. 

D. Head-Mounted Display System 

Referring briefly to FIG. 2, the head-mounted display 
system 14 provides the operator with video feedback of the 
environment at the remote location. Referring now to FIG. 
3, the stereo video signals generated by the stereo cameras 
30, 32 are channeled to the computer 40, which executes a 
graphics algorithm to generate an artificial video image. Hie 
video image then is transmitted by the computer 40 to a 
display driver 80, which activates a helmet-like device 82 
with two display screens, each fed from one of the video 
channels, that depict the remote scene to the operator. 
Alternatively, the video image from the stereo cameras may 
be transmitted to the head-mounted display without use of a 
graphics generated image. In either case, the necessary 
software and equipment, may be purchased commercially. 
One such suitable display system is the Flight Helmet 
manufactured by Virtual Research. 

The preferred embodiment of the present invention also 
includes a microphone 84, which is physically attached to 
the helmet 82 and coupled through a voice recognition 
system 86 to the computer 40. The microphone and voice 
recognition system enable the operator to execute control 
operations (turn on laser, find range to target, move robot to 
target, etc.) without using hands or feet. This feature is 
particularly critical in an application of the invention for the 
benefit of an operator who is required to use his hands for 
other tasks, or for the benefit of handicapped persons, as 
described more particularly below. A commercially avail- 
able voice recognition system suitable for the present appli- 
cation is the DS400 manufactured by Speech Systems, Inc. 
Hie voice command system is operable for generating 
digital data messages within the computer that are sent to 
servo-mechanisms which position the robotic arm as 
instructed, in accordance with techniques known in the art 

E. Position and Orientation Sensor System 

Referring briefly to FIG. 2, the position and orientation 
sensor system 16 detects the position and orientation of the 
operator’s head and communicates the information to the 
computer, as will be described in greater detail, which in turn 
communicates the information to the laser/ camera platform 
control system 18. In this manner, the position and orienta- 
tion sensor system 16 enables die stereo cameras and 
attached laser at a remote location to mimic movements of 
die operator’s head. 

Referring now to FIG. 3, the position and orientation 
sensor system includes a source 90, for generating a position 
sensor and orientation magnetic field in the region of die 
head-mounted display helmet 82 and a position and orien- 
tation sensor 92 mounted on the helmet 82 for generating 
electrical signals representative of position and orientation 
of die helmet within the magnetic field generated by die 
source 90. In die preferred embodiment, a Polhemus “Fast 
Trak” sensor is used as the position and orientation sensor 
92. The system further includes an interface unit 94 that 
receives electrical signals from the sensor 92 and generates 
digital position data (x, y, z) and digital orientation data (roll, 
pitch, yaw) that is communicated to the computer 40. 

The position and orientation sensor system described 
herein is available commercially. Products suitable for use in 
this application include the FastTrack systems manufactured 
by Polhemus or The Bird manufactured by Ascension 
Device. 
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F. Laser/Camera Platform Control System 

Referring briefly to FIG. 2, the laser/camera platform 
control system 18 is a positioning apparatus for moving the 
stereo cameras and (he laser relative in response to motion 
5 of the operator’s head at a remote location. Referring now to 
FIG. 3, the laser/camera platform control system 18 com- 
prises a mechanical platform support structure 190, a roll- 
axis servo motor 102, a pitch-axis servo motor 104, a 
yaw-axis servo motor 106, and a motor controller 108. 
to In the preferred embodiment, the stereo cameras 30, 32 
are affixed to the platform 100 in such a manner that the lines 
of sight of the two cameras are parallel. The laser diode 42 
is also affixed to the platform 100 and aligned such that its 
beam projects along a line that is parallel to the lines of sight 
15 of the stereo cameras 30, 32. The beam of the laser diode 42 
need not lie along a line that is co-planar with the lines of 
sight of the stereo cameras 30, 32. 

The platform 100 has three degree s-of-freedom motion 
capability, with each degree-of-ff eedom driven by one of the 
20 three servo motors 102, 104, 106. The three motors 102, 104, 
106 are controlled by the motor controller 108, which 
receives orientation position commands from the computer 
40. The compute 4D receives data reflecting the position (x, 
y, z) and orientation (roll, pitch, yaw) of the operator’s head, 
25 as described above in the section entitled, “Position and 
Orientation Sensor System,” and transforms the orientation 
data into a set of orientation angles (roll, pitch, yaw) that are 
communicated to the motor controller 108. 

The servo motors 102, 104, 106 preferably comprise 
30 DC-servo motors, such as the type 2842-024C motor manu- 
factured by MicroMo. The motor controller may be, for 
example, a type LM629 controller manufactured by 
National Semiconductor, Inc. 

G. Robot Control System 

35 In the preferred embodiment, the robot 19 (FIG. 2), 
comprises the output device for the telerobotic application of 
the present invention, and the robot arm selected for use in 
connection with the present invention may vary from one 
application to the next Referring now to FIG. 3, the robot 
40 control system typically includes a robot arm 110, compris- 
ing one or more sections, and a robot motion controller 112, 
and a kinematics algorithm executed by the computer 40. 

Referring still to FIG. 2, the robot arm 110 preferably 
includes an end effector, such as a mechanical hand 111 for 
45 accomplishing the purposes of the robot arm. One goal of 
the present invention is to enable the system 10 to command 
the end effector on the robot arm 110 to a precise position 
that corresponds to the position of the target object. The 
computer 40 defines the Cartesian coordinates of a position 
50 command based on information ascertained through the 
ranging algorithm, as described above. Using the kinematics 
algorithm, the computer 40 computes the motion controller 
commands needed to move the end effector on the robot arm 
110 to the target position. The commands are communicated 
55 by the compute 40 to the motion controller 112, which in 
turn controls the motors that move the robot arm 110. One 
robot arm suitable for telerobotics applications is the type 
PUMA 560 manipulator manufactured by Staiibli. 

H. General Operating Procedure of the Preferred Embodi- 
60 ment 

Referring still to FIG. 3, a typical operating procedure 
involves the operator wearing the head-mounted display 82, 
with die orientation sensor 92 attached above the helmet, 
and the microphone 84 positioned near the mouth. The 
65 sensor system source 90 preferably is mounted on a fixture 
attached to the floor. The sensor 92 on the helmet will 
measure the orientation of the operator’s head relative to the 
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source’s reference frame to define a position for the sensor 
92 in three dimensional space. The operator can change die 
position of the sensor 92 by turning or changing the eleva- 
tion of his or her head. This head movement, therefore, is 
analogous to the manipulation of a joystick. Hie orientation 5 
sensor system outputs data to the computer 40, preferably 
via a RS232 serial link. The operator’s head orientation is 
then transformed through the kinematics algorithm of the 
stereo camera platform 108 to obtain joint commands that 
will move the laser/camera platform 100 to the same orien- 
tation relative to the robot’s own reference frame, in accor- 
dance with conventional techniques. The joint commands 
are sent to the laser/camera platform motor controller 108 
via a parallel or RS232 port. 

To acquire the range information of a visual target 
selected by the operator, the operator can simply point die 15 
laser 42 at the target. The reflected laser “dot” can be “seen” 
by both the frame grabbers 34, 36 as well as the operator. 
The frame grabbers 34, 36 capture die video images and 
digitize them. The digital images are stored in memory and 
transmitted to the computer 40 for processing. Through the 20 
frame subtraction technique, the laser “dot” is isolated in 
each of the left and right images. The computer 40 then finds 
the centroid of the laser “dot” in the left and right images 
according to die method of FIG. 5, as discussed above. The 
computer 40 preferably uses the x-axis disparity between the 25 
two centroids to calculate the range of the target using the 
ranging algorithm. 

Because the orientation of the laser/camera platform 100 
and the range of die target are now known, die target’s 
Cartesian position relative to the laser/camera platform 100 30 
can be determined. Once the Cartesian position of the target 
is known, the robot arm 110 can be commanded to move to 
that location by voice command via the microphone 84. 
Once the operator has completed the manipulation of the 
target object, the laser range finding system can be deacti- 35 
vated by voice command. 

L Alternative Embodiment 

The present invention can also be used in applications 
involving operators who must operate other controls, or who 
cannot devote full attention to the robot controller, or who 40 
are physically handicapped. Referring to FIG. 8, a robot arm 
120 can be mounted on a support structure adjacent a 
workpiece. In the exemplary embodiment of FIG. 8, the 
support structure comprises a wheelchair 122, and the opera- 
tor can use the alternative embodiment of the invention to 45 
identify the target location so that the robot can grasp or 
manipulate the target object based on the voice commands 
given by the user. 

In the alternative embodiment of the invention, illustrated 
in FIG. 8, the stereoscopic cameras 124 and laser 126 are 50 
mounted on a helmet 128 worn by the operator along with 
the magnetic direction/position sensors 130. This embodi- 
ment eliminates the need for a laser/camera platform control 
system because the camera and laser will be moved into 
position locally by the operator’s head movements instead of 55 
remotely. 

The present invention allows the operator to specify the 
target of interest virtually hands-free. The hands-free 
method of target selection is more user-friendly to physically 
handicapped persons than those methods that require some 60 
form of manual pointing. As in the first embodiment, com- 
putation is limited by using the frame subtraction algorithm 
with the laser illuminated image and the non-laser illumi- 
nated image. This reduces the memory/computer size and 
power necessary to perform the computation, as well as 65 
allowing the algorithm to run faster than others on a given 
processor. 
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While a preferred embodiment of the invention has been 
shown and described, modifications thereof can be made by 
one skilled in the art without departing from the spirit of the 
invention. 

We claim: 

1. A ranging system for telerobotics applications, com- 
prising: 

a first video camera; 

a second video camera spaced from said first video 
camera, said first and said second video cameras being 
located in a common plane; 

a directional light source positioned relative to said first 
and said second video cameras for illuminating a 
portion of a selected target during selected periods; 

capture means, associated with said video cameras, for 
capturing a first set of stereoscopic images of the target 
while said directional light source is illuminating a 
portion of the target, and for capturing a second set of 
stereoscopic images of the target while said directional 
light source is not illuminating a portion of the target, 
and means for generating a third set of stereoscopic 
images of the target eliminating common elements of 
the first and second set of images; 

a robotic device; and 

a processor electrically associated with said capture 
means and said robotics device, said processor includ- 
ing a ranging algorithm for determining the range to 
said target based upon remaining elements in the third 
set of stereoscopic images. 

2. A system as in claim 1, wherein said video cameras are 
mounted on a camera platform. 

3. A system as in claim 2, wherein said camera platform 
is electrically coupled to said processor, further comprising 
a position and orientation sensor coupled to said processor, 
and wherein said processor positions said camera platform 
based upon electrical signals generated by said position and 
orientation sensor. 

4. A system as in claim 3, wherein said position and 
orientation sensor comprises means for generating signals 
indicative of the movement of an operator’s head. 

5. A system as in claim 4, wherein the operator is 
positioned remotely from said video cameras. 

6. A system as in claim 1, wherein said directional light 
source is activated by an operator remote from said light 
source. 

7. A system as in claim 1, wherein said light source is 
activated by a physically handicapped person. 

8. A system as in claim 1, wherein said directional light 
source illuminates a spot on said target 

9. A system as in claim 8, wherein said means for 
capturing a first set of stereoscopic images of the target 
while said directional light source is illuminating a portion 
of a selected target, and for capturing a second set of 
stereoscopic images of the target while said directional light 
source is not illuminating a portion of the selected target 
include first and second frame grabbers associated with said 
first and second video cameras, respectively, and wherein 
said first frame grabber generates a first image of said target 
with said spot, and said second frame grabber generates a 
second image of said target with said spot, and said proces- 
sor determines the disparity in said plane between the 
position of the spot in said first image and the position of the 
spot in said second image. 

10. A system as in claim 1, wherein said first frame 
grabber is electrically coupled to said first video camera, and 
said second frame grabber is electrically coupled to said 
second video camera, and wherein the output signals from 
said first and said second frame grabbers are provided as 
input signals to said processor. 



5,684,531 


11 

11. A system as in claim 5, further comprising a remote 
display for said operator, said display receiving images from 
said video cameras through said processor. 

12. A system as in claim 1, wherein said directional light 
source is positioned in the same plane as said video cameras. 

13. A system as in claim 1, wherein said processor 
determines the disparity in said plane between the illumi- 
nated image of said first camera and the illuminated image 
of said second camera. 

14. A system as in claim 1, wherein said video cameras lie 
in a horizontal plane with respect to each other, for deter- 
mining horizontal disparity. 

15. A system as in claim 1, wherein said light source 
comprises a laser. 

16. An apparatus for determining range to a target, com- 
prising: 

a directional light source operable to selectively illumi- 
nate a portion of a target to selectively create an 
illuminated spot on said target; 

a first video camera for capturing a first digital image of 
said spot; 

a second video camera for capturing a second digital 
image of said spot; and 

a computer operably associated with said first video 
camera and said second video camera for determining 
the range to said target by comparing said first and said 
digital images to determine disparity, wherein the range 
is dete rmin ed from a look-up table for the determined 
disparity value. 

17. An apparatus as in claim 16, wherein said computer 
obtains the stereoscopic digital images of said spot by 
eliminating common pixels between an image of said target 
without said spot and an image of said target with said spot 

18. An apparatus as in claim 17, wherein said directional 
light source is a laser and wherein said computer turns off 
and on said laser. 

19. An apparatus as in claim 18, further comprising a 
means for directing the directional light source on said 
target 

20. An apparatus as in claim 19, wherein said video 
cameras are mounted on a platform, and said platform is 
positioned by an operator to obtain said images. 

21. An apparatus as in claim 20, comprising means for 
positioning the platform in response to head movement of 
the operator. 

22. An apparatus as in claim 21, further including an 
operator display receiving images from said video cameras. 

23. An apparatus as in claim 22, including a head mounted 
unit, and wherein said operator display is provided in said 
head-mounted unit 

24. An apparatus as in claim 23, further comprising a 
robotics device connected to said computer, wherein said 
operator activates said robotics device and said computer 
provides an electrical signal indicative of target location to 
said robotics device. 

25. An apparatus as in claim 24, wherein said robotics 
device is activated by a voice command from the operator. 

26. An apparatus as in claim 25, further comprising a 
voice recognition system connected to said computer, said 
voice recognition system comprising means for converting 
said voice command to an electrical output signal. 

27. A method for determining range to a target, compris- 
ing the steps of: 

(a) capturing a first set of stereoscopic images of a target; 

(b) activating a directional light source for illuminating a 
portion of said target; 

(c) capturing a second set of stereoscopic images of the 
target while the light source is directed onto the portion 
of said target; 
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(d) generating a third set of stereoscopic images of the 
target by eliminating common elements of the first and 
second set of images; and 

(e) determining range to the target based upon remaining 
5 elements in the third set of stereoscopic images. 

28. A method as in claim 27, wherein the range to the 
target is determined by calculating the disparity between the 
remaining elements of the third set of stereoscopic images. 

29. A method as in claim 27, wherein said stereoscopic 
10 images includes a left video image and a right video image. 

30. A method as in claim 29, wherein said video images 
are digitized. 

31. A method for remotely controlling a robotics device to 
manipulate a target device, comprising the steps of: 

15 (a) remotely viewing the target device; 

(b) capturing a left and a right video image of said target 
device; 

(c) activating a light source to generate a spot on said 
20 target device; 

(d) capturing a left and a right video image of said target 
device with said spot thereon; 

(e) eliminating common pixels between said video images 
captured in step (b) and said video images captured in 

25 step (d) to obtain a left and right image of said spot; 

(f) determining disparity between said left and right image 
obtained in step (e); 

(g) calculating range to said target device based upon said 
disparity determination; 

30 (h) positioning said robotics device to contact said target 

device based upon said range calculation. 

32. A method as in claim 31, wherein the step of remotely 
viewing the target device includes positioning a left and 
right video camera to view the target device. 

35 33. A method as in claim 32, wherein the positioning of 

said left and right cameras is performed by moving the head 
of an operator. 

34. A method as in claim 31, wherein the video images are 
captured by a left and right frame grabber circuit. 

40 35. A method as in claim 31, wherein said laser is 

positioned by an operator and activated by said operator. 

36. A method as in claim 31, wherein said laser is 
activated by a computer after said target device is selected 
by an operator. 

45 37. An apparatus for determining range to a target, com- 

prising: 

a directional light source for illuminating a portion of a 
selected target during selected periods; 

means including first and second video cameras for cap- 
50 turing a first set of stereoscopic images of the target 
while said directional light source is illuminating a 
portion of a selected target, and for capturing a second 
set of stereoscopic images of die target while said 
directional light source is not iUuminating a portion of 
the selected target, and for generating a third set of 
55 stereoscopic images, of the iUuimnated portion of the 
target, by eliminating common elements of the first and 
second set of images; 

a computer operably associated with said first and second 
video cameras and said robotics device for determining 
60 the range to said target by comparing the positions of 
the illuminated portion of the target within said third set 
of images, based upon elements remaining in the third 
set of stereoscopic images and with common elements 
in said first and second sets of stereoscopic images 
65 eliminated, to determine disparity. 


* * * * * 



